{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "display(HTML(\"<style>.container { width:90% !important; }</style>\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lecture 3C - Group By, Aggregation Operations and Shaping Data*\n",
    "---\n",
    "\n",
    "\n",
    "### Content\n",
    "\n",
    "1. Group-by operations\n",
    "2. Pivot tables\n",
    "3. Cross tabulation\n",
    "\n",
    " \n",
    "\n",
    "\\* Content in this notebook is based on the material in the \"Python for Data Analysis\" book by Wes McKinney, chapter 9.\n",
    "\n",
    "### Learning Outcomes\n",
    "\n",
    "At the end of this lecture, you should be able to:\n",
    "\n",
    "* explain the group by mechanism\n",
    "* split pandas objects into groups using one or more keys\n",
    "* compute group summary statistics\n",
    "* perform cross tabulation\n",
    "* generate pivot tables\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import HTML, IFrame\n",
    "IFrame(\"http://pandas.pydata.org/pandas-docs/dev/groupby.html\", width=1100, height=500)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Categorizing a data set and applying operations to each group, be it **transformations or aggregations, is frequently a critical component of a data analysis workflow**. \n",
    "\n",
    "After loading, merging, and preparing a data set, a **common task is to compute group statistics or possibly pivot tables for reporting or visualization purposes**. \n",
    "\n",
    "**Pandas provides a flexible and high-performance groupby facility**, enabling you to slice and dice, and summarize data sets in a natural way.\n",
    "\n",
    "One reason for the popularity of relational databases and SQL (which stands for “structured query language”) is the **ease with which data can be joined, filtered, transformed, and aggregated**. However, query languages like **SQL suffer from limitations in the kinds of group operations that can be performed**. \n",
    "\n",
    "With the expressiveness and power of Python and pandas, **much more complex grouping operations can be executed** by utilizing any function that accepts a pandas object or NumPy array.\n",
    "\n",
    "Most off-the-shelf software like Excel can to some degree perform operations like pivot tables that are immensely valued. However, Excel has serious limitations. The greatest of these is memory. Even that latest Excel suite can only handle up to 1M rows but not more (using the PowerPivot add-on, this can now be extended). Pandas provided the **ability to flexibly handle millions of rows of spreadsheet data up to the maximum capacity of your PC's on board RAM**. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Group-By operations\n",
    "\n",
    "One of the most powerful features of Pandas is its **GroupBy** functionality. On occasion we may want to perform operations on *groups* of observations within a dataset. For example:\n",
    "\n",
    "* **aggregation**, such as computing the sum of mean of each group, which involves applying a function to each group and returning the aggregated results\n",
    "* **slicing** the DataFrame into groups and then doing something with the resulting slices (*e.g.* plotting)\n",
    "* group-wise **transformation**, such as standardization/normalization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A common data analysis procedure is the **split-apply-combine** operation, which **groups subsets of data together, applies a function to each of the groups, then recombines them into a new data table**.\n",
    "\n",
    "For example, we may want to aggregate our data with with some function.\n",
    "\n",
    "![split-apply-combine](http://f.cl.ly/items/0s0Z252j0X0c3k3P1M47/Screen%20Shot%202013-06-02%20at%203.04.04%20PM.png)\n",
    "\n",
    "<div align=\"right\">*(figure source \"Python for Data Analysis\", p.252)*</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "from pylab import rcParams\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set some Pandas options as you like\n",
    "pd.set_option('max_columns', 40)\n",
    "pd.set_option('max_rows', 30)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rcParams['figure.figsize'] = 15, 10\n",
    "rcParams['font.size'] = 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame({'key1' : ['a', 'a', 'b', 'b', 'a'],\n",
    "                   'key2' : ['one', 'two', 'one', 'two', 'one'],\n",
    "                   'data1' : np.random.randn(5),\n",
    "                   'data2' : np.random.randn(5)})\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Suppose you wanted to compute the mean of the data1 column using the grouped labels\n",
    "from key1. There are a number of ways to do this. One is to slice data1 and call\n",
    "groupby with the column (a Series) at key1:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grouped = df['data1'].groupby(df['key1'])\n",
    "grouped"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This grouped variable is now a GroupBy object. It has not actually computed anything\n",
    "yet except for some intermediate data about the group key df['key1']. \n",
    "\n",
    "The idea is that this object has all of the information needed to then apply some operation to each of\n",
    "the groups. \n",
    "\n",
    "We can find out what groups we have and who is in these groups by index:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grouped.groups"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Iterating over groups\n",
    "\n",
    "In order to inspect which rows have been associated with grouping values, we need to iterate through each group."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for key, value in grouped:\n",
    "    print(key, type(value))\n",
    "    print(value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have our groupings, we can compute operations on groups using various pre-defined methods or built in methods. Here is the example of applying means to each group:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grouped.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could have taken a shortcut and calculated the mean and returned a Series object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['data1'].groupby(df['key1']).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or alternatively:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[['data1','key1']].groupby('key1').mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise**: Perform group by on 'key2' and calculate the sum of values on the 'data2' column. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise:** Using df above, display the result of a group by operation on key1, showing the count/number of values that have been aggregated:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We sometimes want to perform group by operations on all columns in  a data frame, whilst grouping the data on one or more columns. This can be done as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gr = df.groupby('key1').mean()\n",
    "gr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that the returned object in this instance is a data frame since we made use of all columns, and that 'key2' column is ignored since a mean on it cannot be performed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Applying multiple operations on groups\n",
    "\n",
    "\n",
    "Pandas comes with a number of built-in and optimised operations which can be applied simultaneously to the groups using the  `agg` function.\n",
    "\n",
    "Optimized groupby methods:    \n",
    "\n",
    "**Function name Description**\n",
    "- count = Number of non-NA values in the group\n",
    "- sum = Sum of non-NA values\n",
    "- mean = Mean of non-NA values\n",
    "- median = Arithmetic median of non-NA values\n",
    "- std, var = Unbiased (n - 1 denominator) standard deviation and variance\n",
    "- min, max = Minimum and maximum of non-NA values\n",
    "- prod  = Product of non-NA values\n",
    "- first, last = First and last non-NA values\n",
    "\n",
    "- In addition we can define our own functions to apply to groups"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def my_length(x):\n",
    "    return len(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grouped = df.groupby('key1')\n",
    "result = grouped.agg(['sum','count','mean','median','min','max','std','var', my_length])\n",
    "result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As seen from the above example, we can define our own functions and apply them to the groups:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grouped.apply(my_length)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grouped"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Grouping on multiple columns\n",
    "\n",
    "We could also perform group by on multiple columns:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grouped = df.groupby(['key1','key2'])\n",
    "grouped.groups"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that in this case we do not end up with a simple index. What we end up with here is what is called a **hierarchical index**. Each row then becomes accessible using the combination of the two indexes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grouped.groups['a','one']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use these index values to access the actual values in the original dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.iloc[grouped.groups['a','one']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now create a new dataframe that has the mean values of all the groups"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_grouped = grouped.mean()\n",
    "df_grouped"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make a quick plot of this data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_grouped.plot(kind='bar')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Shaping and reshaping data with **hierarchical indexes**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hierarchical indexes are created when we group data frames using multiple columns. They are very useful as they allow us to reshape our data.\n",
    "\n",
    "Consider the dataframe object which was generated from the example above:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_grouped"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to  reshape data (hierarchically structured dataframes) we have functions **`stack`** and **`unstack`** available to us.\n",
    "\n",
    "        stack: “pivot” a level of the (possibly hierarchical) column labels, returning a DataFrame with an index with a new \n",
    "        inner most level of row labels.\n",
    "        \n",
    "        unstack: inverse operation from stack: “pivot” a level of the (possibly hierarchical) row index to the column axis, \n",
    "        producing a reshaped DataFrame with a new inner-most level of column labels.\n",
    "\n",
    "The above dataframe is already `stacked` which is indicated by the second index indicating hierarchical indexing. We can stack it one further level by taking the columns and making them into columns:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "stacked_df = df_grouped.stack()\n",
    "pd.DataFrame(stacked_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_grouped"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, we can unstack the original dataframe by taking the inner-most index ('key2') and moving it into the column axis:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "unstacked_df = df_grouped.unstack()\n",
    "unstacked_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_grouped"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can slice into dataframes and perform stacking/unstacking on selected columns:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = df_grouped['data1']\n",
    "s"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above is now a series object having a hierarchical index which we can now reshape into a data frame with the help of `unstack` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df2 = s.unstack()\n",
    "df2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise**: Stack the df_grouped dataframe so that data1 and data2 now become rows and store this result in a new dataframe. Now unstack this new dataframe by making key1 into columns. Look up the function to see how you can specify which row to unstack explicitly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Removing hierarchical indexes from group by operations\n",
    "\n",
    "Sometimes we do not want to preserve the hierarchical index. We can stop the group by operation from generating it by setting the as_index argument to False:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_grouped_mean = df.groupby(['key1','key2'], as_index=False).mean()\n",
    "df_grouped_mean"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.groupby(['key1','key2']).mean().reset_index()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Pivot Tables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A **pivot table is a data summarization tool** that is very much like the group by facility (and is actually powered by pandas group by under the hood). This tool/operation is routinely performed in spreadsheet programs and other data analysis software. It aggregates a table of data by one or more keys, arranging the data in a rectangle with some of the group keys along the rows and some along the columns. Pivot is often regarded as one of Excels greatest strengths.\n",
    "\n",
    "Pandas' groupby facility enables pivot table functionality, together with the reshape operations utilizing hierarchical indexing seen earlier. \n",
    "\n",
    "DataFrame has a  dedicated pivot_table method, and additionally there is a top-level  pandas.pivot_table function. \n",
    "\n",
    "To illustrate these features, a less trivial dataset on restaurant tipping will be used called \"tips.csv\". It is from the R  reshape2 package; it was originally found in Bryant & Smith’s 1995 text on business statistics (and found in the book’s GitHub repository)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_tips = pd.read_csv(\"../datasets/tips.csv\")\n",
    "df_tips.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Suppose we wanted to compute a table of group means (the default  pivot_table aggregation type) arranged by  sex and  smoker on the rows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_tips.pivot_table(index=['sex', 'smoker'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This could have been easily produced using  groupby as well.\n",
    "\n",
    "**Exercise:** Perform the above using groupby."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, as the perspective which you would like to view data from becomes more complex, additional constructs become helpful.\n",
    "\n",
    "Now, suppose we want to aggregate only tip_percent and table-party size and we want these aggregations to be split between those who are smokers and non-smokers. And suppose we want to additionally group by day as well as sex. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_tips.pivot_table(values=['tip_percent', 'size'], index=['sex', 'day'], columns='smoker')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise:** Create a pivot table that aggregates by mean only tip_percent and we want these aggregations to be split between dinner and lunches. And  we would like to group by sex and smoker. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Mean is the default operation performed with pivot tables. Other functions can be used, for example 'len'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_tips.pivot_table(values='tip_percent', index=['sex', 'smoker'], columns='day', aggfunc=len, margins=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise:** Create a pivot table that sums up the total bills for each daya and splits them by smoker."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Cross-Tabulation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A cross-tabulation (or crosstab for short) is a special case of a pivot table that computes\n",
    "group frequencies."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first two arguments to  crosstab can each either be an array or Series or a list of\n",
    "arrays. As in the tips data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.crosstab([df_tips.time, df_tips.day], df_tips.smoker)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise:** Cross-tabulate the tips data in order count the number of tips for dinner and lunches, while grouping by sex and smoker and showing all the margins too. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercises\n",
    "\n",
    "Exercises below will use a geographic dataset sourced from http://www.geonames.org/countries/ called \"country_info.csv\".\n",
    "\n",
    "Load the above dataset into a dataframe and inspect the data.\n",
    "\n",
    "Use this dataset to answer the exercises below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise** Group the dataset by continent, showing the total population in each one. Plot the result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise** Group the dataset by CurrencyName, showing the total surface Area that each one covers. Sort the result by size and display the top ten currencies by land surface area. Plot the result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise** Perform the same aggregation as above, this time using Population instead of land Area."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise** Use the pivot table to display information for currencies in Europe only. Group the data by continent and currency name, showing only the results from Europe. Aggregate by using summation for the Area and the Population variables and show the margins."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise**: Cross-tabulate the dataset to find out the counts for currency names across Europe and Americas together with the margins."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise**: On the Stream site, you will find links to a number of repositories. \n",
    "\n",
    "1. Go through these repositories and identify two datasets which you can practice merging and concatenating on.\n",
    "2. Apply appropriate group by operations on this data in order to draw out insights\n",
    "3. Apply combinations of appropriate group by operations as well as pivots and cross-tabulation on this data in order to draw out insights."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
